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ABSTRACT 

Ubiquitous Learning Log (ULL) is defined as a digital record of what you have learned in the daily life using ubiquitous 
technologies. This paper focuses on how to capture learning experiences in our daily life for vocabulary learning. In our 
previous works, we developed a system named SCROLL (System for Capturing and Reminding Of Learning Log) in 
order to log, organize, recall and evaluate the learning log. However up to now, we just use an active mode to record logs. 
This means that a learner must take a capture of learned contents consciously and most of learning chances be lost 
unconsciously. This paper proposes a system named PACALL (Passive Capture for Learning Log) in order to have a 
passive capture using SenseCam to solve this problem. With the help of SenseCam, learner’s activity can be captured as a 
series of images. With the help of this system, a learner can find the important images by analyzing sensor data and 
images processing technology. 
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1. INTRODUCTION 

CSUL (Computer Supported Ubiquitous Learning) or context-aware ubiquitous learning (u-Learning) is 
defined as a technology enhanced learning environment supported by ubiquitous computing such as mobile 
devices, RFID tags, and wireless sensor networks (Ogata et al, 2004). CSUL augments learning in the real 
world by presenting information on personal mobile devices through the Internet and surrounding 
environment like physical objects and sensors. Those CSUL applications are intended to be used all the time. 
This is one of the advantages CSUL called permanency. It means that learners never lose their work unless it 
is purposefully deleted and all the learning processes are recorded continuously every day. However, little 
attention has been paid to this aspect despite much attention being paid to other features such as accessibility, 
immediacy and interactivity to the Internet, physical environment and other learners. 

The fundamental issues of CSUL are: 

(1) How to record and share learning experiences that happen at anytime and anyplace. 

(2) How to retrieve and reuse them in future learning. 

To tackle those issues, LORAMS (Linking of RFID and Movie System) (Ogata et al. 2007) was 
proposed. There are two kinds of users in this system. One is a provider who records his/her experiences into 
videos. The other is a user who has some problems and is able to retrieve the videos. The system 
automatically links between physical objects and the corresponding objects in a video and allows sharing 
them among users. By scanning RFID tags, LORAMS shows the user the video segments that include the 
scanned objects. Although this system is useful in certain environments, it is not easy to be applied in 
practice at any place at the moment. Therefore, we started more practical research called “ubiquitous learning 
log (ULL)” project in order to store intentionally what we have learned as ubiquitous learning log objects 
(ULLOs) and consequently reuse them. 

We defined ubiquitous learning log (ULLO) as a digital record of what a learner has learned in the daily 
life using ubiquitous technologies and proposed a model called LORE to show the learning processes in the 
perspective of the learner’s activity. In this paper, we propose a system SCROLL (System for Capturing and 
Reminding Of Learning Log) that helps learners log their learning experiences with photos, audios, videos, 
location, QR-code, RFID tag and sensor data and share ULLOs with others. Also, learner can receive 
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personalized quizzes and answers for their question (Ogata, et al., 2010). This system is implemented both on 
web and android smartphone platforms. With the help of built-in GPS and camera on smartphone, learners 
can navigate and be aware of past ULLOs by augmented reality view. 


2. LIFE-LOG 

Life-log is a notion that can be traced back at least 60 years (Bush, 1945). The idea is to capture everything 
that ever happened to us, to record every event we have experienced and to save every bit of information we 
have ever touched. For example, SenseCam (Hodges et ah, 2006) is a sensor augmented wearable stills 
camera; it is proposed to capture a log of the wearer’s day by recording a series of images and capturing a log 
of sensor data. This is a great tool for recording life log. is a small digital camera that is combined with a 
number of sensors to help to capture a series of images of the wearer’ s whole daily life at the proper time and 
it can be worn around the neck (Figure 1). Originally this device is designed for memory aid. MyLifeBits 
(Gemmell, Bell, & Lueder, 2006) stores scanned material (e.g.: articles, books) as well as digital data (e.g.: 
emails, web pages, phone calls, and digital photos taken by SenseCam). Ubiquitous Memory system 
(Kawamura, Fukuhara, Takeda, Kono, & Kidode, 2007) is a life-log system using a video and RFID tags. 
Also, Evemote (www.evernote.com) is a tool to save ideas using mobile devices such as Android and iPhone. 
The most common idea of those projects is to use life-log data for memory aid. SCROLL, however, aims to 
utilize life-log data for the learning process. 



Figure 1. SenseCam 


3. SCROLL: UBIQUITOUS LEARNING LOG SYSTEM 
3.1 Design 

Learning Log was originally designed for children as a personalized learning resource (Ogata et al. 2010a). It 
was set by teachers to help their students record their thinking and learning. In this learning log, the logs were 
usually visually written notes of learning journals. How are we learning from past learning log? For example, 
we take notes, e.g., vocabularies, idioms, sentences in a language learning situation. Whereas, they will not 
remind us of the knowledge learned, nor the situation where the knowledge was used. We think this process 
can be enhanced using mobile devices. We proposed learning processes in the perspective of the learner’s 
activity model called LORE (Log-Organize-Recall-Evaluate). 

(1) Log what the learner has learned: When the learner faces a problem in the daily life, s/he may learn 
some knowledge by her/himself, or ask others for a help in terms of questions. The system records what s/he 
learned during this process as a ULLO. 

(2) Organize ULL: When the learner tries to add a ULLO, the system compares it with other ULLOs, 
categorizes it and shows the similar ULLOs if exist. By matching similar objects, the knowledge structure 
can be regulated and organized. 

(3) Recall ULL: The learner may forget what s/he has learned before. Rehearsal and practice in the same 
context or others in idle moments can help the learner to recall past ULLOs and to shift them from short-term 
memory to long-term one. Therefore, the system assigns some quizzes and reminds the learner of her/his past 
ULLOs. 
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(4) Evaluate; It is important to recognize what and how the learner has learned by analyzing the past 
ULL, so that the learner can improve what and how to learn in future. Therefore, the system refines and 
adapts the organization of the ULLOs based on the learner’s evaluation and reflection. 

Since 2009, we started our project and developed a system named SCROLL (System for Capturing and 
Reminding Of Learning Log) (Ogata et al. 2010a, 2010b) that helps learners collect their learning 
experiences as ubiquitous learning objects (ULLOs). Also, all of the collected ULLOs are organized, shared 
in this system, and the learning effect can be enhanced. 

3.2 Learning Types 

We designed SCROLL as a model of system to implement to implement the following types of learning: 

(1) Self-directed and personalized learning 

The first one is self-directed and personalized learning. We design SCROLL based on these two 
objectives that adopt self-directed and personalization: 

(a) By being aware of a learner’s current context, especially the location information, the system can 
detect whether a learner is near to the place where he uploaded a learning log and whether there are location- 
based learning logs recorded by other learners close to him. If either requirement is met and the availability 
of the device is high, the system will show him a quiz based on the knowledge he gained around there or 
notify him the surrounding learning logs added by others. 

(b) The system can record the context data when a learner uses the system to study as his context history 
and then catches his learning habits by making use of the context history. If the learning habits exist and the 
circumstance meets the learning habits, the system will show a piece of recommendation message to 
encourage him to review what he has learned. 

(2) Reflective learning 

An important goal of SCROLL system is to help learners recall what they have learned after they 
archived their learning logs. When a learner captures his learning log, besides the location based property 
mentioned above, a number of things are designed for learners to encode as retrieval cues. Lor instance, 
according to the picture superiority effect, the learning logs with pictures are much more likely to be 
remembered rather than those without pictures. In addition, according to the basic research on human 
learning and memory, practicing retrieval of information (by testing the information) has powerful effects on 
learning and long-term retention. And compared with repeated reading, repeated testing enhances learning 
more. Lor these two reasons, the quiz function taking advantages of the pictures, locations and so on is 
proposed. Three types of quizzes can be generated automatically by the system, which are yes/no quiz, text 
multiple-choice quiz and image multiple-choice quiz. Usually, learners can examine themselves by practicing 
the quizzes. But two more ways that are provoked by the system are provided. One is that when a learner 
moves to the place where he took down knowledge, the system can show quizzes about the learned 
knowledge for him. The other one is that if a learner has his learning habits, the system will prompt him to 
review what he learned in quizzes. In the rest part of this paper, we will talk about them in detail. 

(3) Collaborative learning 

We design SCROLL also as a collaborative learning. Learning log is a log that a learner has done, 
therefore for collaborative learning in SCROLL is asynchronous model. Any learner in this system is able to 
share ULLOs, and system will show the shared ULLOs to others. Besides, they can also ask for others 
questions when they share ULLOs. In reflective learning, shared ULLOs can also be used to generate quizzes 
in order to help learners learn more objects. 

(4) Situated learning and experiential learning 

We design a concept called task in SCROLL to implements the situated learning and experiential learning. 
Tasks are referred as the activities that the knowledge can be used. They are related to the learning contexts 
like school, hospital, post office and so on. Lor instance, if the system recommends a learner a Japanese word 
“ h T h (tomato)” in a supermarket, the learner can talk with the staffs in the supermarket using the word “ h 

h (tomato)”, such as asking its price, location, recipe and so on. And it has been proved that by talking 
with the Japanese native speaker using the recommended word, learner can master the word well. The 
activity of asking about the information is a kind of the so-called task. Basically, the learners who saved the 
learning log are responsible for providing what kinds of tasks the knowledge can be utilized. And one 
learning log can be used in several tasks. Moreover, the system provides some predefined tasks in different 
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contexts in order to reduce the learners’ burden of designating tasks when they save their learning logs. Table 
1 shows part of the predefined tasks in different contexts. What’s more, the tasks can be defined by the 
learner and designated by the administrator of the system. The system assigns the appropriate task for a 
learner based on the difficulty of the task and the learner’s ability. For example, asking the price of the 
production is easy for learners to finish while asking about the recipe of the vegetables is quite difficult for 
most learners. And when the learners received the recommended the learning log and the task, they are also 
asked to provide feedback for the system. For example, they are asked to take the photos of the object if they 
are asked to inquire the location of it. And if they are asked to learn about the place of the production, they 
need to accomplish this information on the system. Only providing the feedback can prove that they have 
really used the knowledge. And if the learner meets new problems when he carries out the tasks, he can 
record them in photos, videos, audios or texts and upload them to the system in order to ask for help. Such 
accumulated data is also meaningful for the other learners. 

(5) Seamless learning 

We design the following Seamless Mobile-Assisted Language Learning Support System (hereafter we 
call it SMALL System) as a sub-project. It contains following parts: 

(1) Textbook Data: consists of the whole units of the textbook to be learned through one semester. A 
teacher uploads PDF file textbook data to the system in advance. 

(2) Learning Log System: or SCROLL is a system developed by our team. Users register what they have 
learned, which we call “learning log objects (LLO) to the system and view LLOs uploaded by themselves 
and others, then it supports recalling of their learning logs by giving them quizzes. 

(3) Quiz: The students register textbook target words and their newly acquired words during their self- 
learning and the system gives them quizzes. It generates quizzes based on the LLOs registered and viewed by 
the students. 

(4) Message: Users can send messages to other users in this system. When a viewer clicks the author 
name of the LLO, new window will be popped up and can send a message to him. This function will promote 
the students interaction or discussion and will lead to collaborative learning which will be inevitable where 
the teacher in not there outside-class self-learning. 

3.3 Interface 

SCROLL is a client-server application, which runs on different platforms including Android mobile phones, 
PC and general mobile phones. It contains the following components: 

(1) ULL recorder: This component facilitates the way for the learners to upload their ULLOs to the server 
whenever and wherever they learn. Learners can take its photo, video and/or voice and ask questions about it 
and attach different kinds of meta-data with it, such as its meanings in different languages, comments, tags 
and location information. Also the learner can enter its barcode and/or RFID and select whether the new 
ULLO can be shared or not. Figure 2(1) is the interface of registering a new learning log that runs on 
Android smartphone. Figure 2(2) is an example of learning log. 

(2) ULL finder: If learner registers a new ULLO, the system checks whether the same object has been 
already stored or not by comparing the name fields of each object using a thesaurus dictionary. Also, the 
learner can search ULLOs by name, location, text tag and time. Using this function, learners can understand 
what, where and when they learned before. In the future works, the visualization of the ULLOs will be 
developed. 

(3) ULL reminder: This function is used to implement the Recall ULL in LORE model. This system 
provides a personalized and context-aware quiz to remind learners of past ULLOs. Quizzes are generated by 
system automatically from the ULLOs registered by all learners. The quiz function is designed not only to 
help the learners to practice what they have learned, but also to recommend what the other learners have 
learned and to remind them to re-learn their past knowledge according to their current location and their 
preferred time. Figure 2(3) is the interface of quiz function on Android. 

(4) ULL navigator: It provides mobile augmented reality that allows the learners to navigate through the 
ULLOs. Like Wikitude and Sekai-Camera, it provides them with a live direct view of the physical real-world 
environment augmented by a real time contextual awareness of the surrounding objects. While a learner is 
moving with his mobile phone, the system shows an alert on the phone as soon as he enters the region of 
ULLOs according to the GPS data. This view is augmented, associated with a visual compass, and 
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overlapped by the nearest objects in the four cardinal directions (Figure 2(4)). It also provides him with a list 
of all surrounding objects. When he selects one or more of these objects, the Google map will be retrieved, 
and marked with his current location and the selected object. Moreover, the system shows a path (route) for 
him to reach to its locations. This assists him to acquire new knowledge by discovering the existed ULLOs 
and to recall his own ULLOs. 



Figure 2. SCROLL on Android. 


4. PCALL 
4.1 Design 

Until now all works that we have done are using active logging mode, not passive logging mode. It means 
that learners must record their learning experiences as learning material consciously. Comparing to the 
passive mode, in the active mode we are more likely to miss learning chances since we are not necessarily 
able to record what we have learned or sometimes we just forget to record it. Therefore, we planned to 
introduce passive capture in our project with SenseCam and named the proposed system as PACALL 
(PAssive Capture for Learning Log). For example in the real world, there are so many things that we have 
learned but we usually miss the change to review them that is we do not know what we know. Similarity, it is 
certain that we are not able to know what we have not noticed. Therefore, we considered over this learning 
process. 

Since this research is based on our previous works that use active mode to register ULLOs, we have to 
find out differences between active mode and passive mode in this research. We have compared both on 
features as Table 1 shows. 


Table 1. Comparison between active mode and passive mode 



Passive 

Active 

Number of photos a day 

Many (~3000/day) 

Few (<10/day) 

Quality 

Low 

High 

Pros 

Avoid to forget taking photos 

Learning actively (intentional learning) 

Cons 

Difficult to find good photos from many photos 

Forget taking photos 


Figure 3 explains this process and shows how to support learning in passive mode. We classify all the 
objects surrounding us into 4 groups - there are “(I) I know what I know”, “(II) I know what I don’t know”, 
“(III) I don’t know what I know” and “(IV) I don’t know what I don’t know”. For example, for non-English 
speaker, when a learner walk outside and see a fire hydrant, if he notice it and remember how to speak it 
English that is the status (I). If he does not know how to speak it in English and that is status (II). Since he 
have notice it, and does not know how to speak it, he can learn it by dictionary or ask for someone else. Then 
this knowledge will be transferred from (II) to (I). This is the process C - learning. It only happens 
consciously that is to say active mode. 
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Another situation, if he has not noticed the fire hydrant how can he learn it? The answer is no way 
without any assist method. Therefore, we want to use life-log as a passive way to support it. There are also 
two situations. First is he have already known how to speak it in English (status III). In this case, captured 
life-log photos can help him notice this fire hydrant and let me revise it. In another case (IV), captured life- 
log photos let him know there is an object that he does not know, and then he can have a chance to learn this 
object (B to C). This is a good way to help a learner know what he/she does not know if he/she does not 
know what he/she does not know. 

Figure 3 is the flow of PACALL in analyzing captured photos. There are 5 steps: 

1 . Loading raw data. 

2. Filtering bad photos. 

3. Finding good photos. 

4. Photo recommendation. 

5. Learning Analytics 
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Figure 3. Knowledge transition. 

Figure 4. Interface of PACALL. 


There are four steps in PACALL. 

(1) Loading raw data 

There are 3 types of raw data in PACALL: life - log photos. Sensor data, and GPS data. Life-log photos 
are captured by SenseCam at the present. In the future, we plan to apply this system to common photos. That 
will be interesting and useful. Imaging that you have tour and took many photos, and then you can use this 
system to find out learning content. Sensor data is record by SenseCam and GPS data is created by portable 
GPS unit. 

(2) Filtering bad photos 

Before filtering bad photos, we must define the bad photo and good photo. In this research, we define that 
bad photo is a photo that is hardly to recognize its content or that is duplicated with other photos, while good 
photo is a photo that contains clear objects. We define three types of bad photos: 

(a) Dark: Dark means a photo taken with insufficient light and the photo is dark. 

(b) Duplicate: Duplicate means the photos are duplicated. 

We use image processing to find out these bad photos. Currently, we are using OpenCV to detect photos 
dark, and using LIRE(a plugin for Lucene) to detect duplicated photos. 

(3) Finding good photos 

As it is defined in (2), the good photo is clear photo and contains clear objects. Therefore, we are using 
OpenCV to find out good photos mainly by feature detection. After finding out good photos, left photos are 
common ones. Those photos are not so clear but maybe contain learning contents. However the priority of 
those photos is lower than good photos when shown to learner to choose. 

(4) Photo recommendation 

Until now, bad photos are filtered out and system selected several good photos for learner. Photo 
preparation is finished, and it steps into the stage of learning content assistant. We attempt to abstract useful 
information from photos by machine, and recommend photos that contain information. Therefore we define 4 
types of recommended photos: 
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(a) Character photo: Character photo means a photo that contains characters. These characters are 
probable used as learning content. Here we are using text detection to find these photos. 

(b) Face photo: It is certain that face photo means a photo that contains faces. Actually, these photos are 
usually not appropriate for learning content because of privacy issues. Anyway, faces are also information 
from photos. 

(c) Taggable photo: Taggable photo means a photo that can be tagged by text. Tag is important 
information of the photo and it is probably used as title of photo. 

(d) ULLO-like photo: If there is a similar photo that was already registered to the SCROLL as a ULLO, 
maybe this photo is also can be used as a ULLO. 

4.2 System Interface 

In the section 4.1, we introduced our research design especially design of the flow of analyzing photos. We 
developed functionalities in detail. 

(1) PACALL Uploader 

PACALL Uploader helps learner upload all the photos after capturing. We want to make it easy to upload 
all the captured photos to the server. Because of the limitation of web technology, this process is not so easy 
in the past. However with HTML5, it becomes possible. When a learner wants to upload the whole folder, 
learner can select a photo folder and upload all the photos to the server. Also, the file of sensor data and GPS 
data will also be uploaded. 

(2) PACALL Browser 

After uploading the raw data (photos, sensor data and GPS data), system will analyze all the data and 
show the result for learner When all the photos are uploaded to the server, the learner can have a reflection of 
all the photos with the help of PACALL. PACALL Browser is an interface of browsing all the photos, and it 
tags photos and provides some information of photos to help learner find important photos (Figure 4). 
Currently, we provide three main functionalities in PACALL Browser - PACALL Filter, PACALL Searcher 
and PACALL Recognizer. PACALL Filter classifies all the photos into categories such as Manual, Normal, 
Duplicate, Dark, Face and Recommendation. Here manual means that photo is captured by pressing manual 
button of SenseCam. It usually happened when learner finds something valuable to record. Duplicate and 
Dark contains bad photo. Face means the photos contains faces and Recommendation includes Manual, Faces 
and other good photos that contains information or have similar photos that have been uploaded to SCROLL 
before. Such photos have tag under the photo like 3d or 4d means there are photos uploaded to system 3 or 4 
days ago. 

(3) PACALL Recaller 

When a learner clicks one photo in PACALL browser, the PACALL recaller will be opened. The photo 
and the similar photos and sensor data will be shown on this page to help user recall the captured content. 
There is also a “Upload” button on this page. If the learner decide to upload this photo to SCROLL as a 
ULLO, s/he can clicks this button, the photo will be uploaded to the SCROLL system directly and the page 
will jump to the learning log registration page (Figure 4). Figure 4 is the interface of ULLO registration in 
SCROLL system. On this page, a learner can see the location of the selected photo and other similar photos 
that captured by SenseCam. If there are some similar photos that are already existed in SCROLL, the similar 
photos will be also shown on this page. Once “Upload Now” is clicked, the system will ask student to answer 
a survey that let system know whether learner know it and noticed this object when it was captured. The data 
can be used to evaluate our system and help learners analyze the learning situations. When an object is 
uploaded to the system, SCROLL system will use “organize”, “recall” and “evaluate” model to help learner 
remember uploaded objects and vocabularies. For example, if a learner uploaded a photo and set the title as 
in Japanese, but s/he does not know how to speak it in English, then s/he can send a question along 
with the uploaded ULLO. SCROLL will send this question to all Chinese users. After receiving the answer 
from Chinese users, this learner learned a new Chinese word. In the quiz module of SCROLL, learner can 
answer the quizzes that are generated by uploaded ULLOs. By answering these quizzes, learner’s knowledge 
will be enhanced. 


123 



ISBN: 978-972-8939-81-6 © 2013 lADIS 


5. CONCLUSION 

In this paper, we discussed how we can learn vocabulary from the life-log pictures. In order to do it, we used 
SenseCam to capture life-log passively and developed a system named PACALL to help learner to register 
learning log objects with vocabulary. We have designed a model of learning process in passive capture mode 
including capture, reflect, store. The PACALL system has been also developed in order to support reflection 
and reduce the workload of reviewing photos. During this research, we found that the SenseCam that 
originally designed for memory aid can be also used to capture learning log for passive mode to help learners 
to learn vocabulary. However, it usually takes too many photos, and many of them are duplicated or dark. 
Therefore, we must introduce other technology to help learners find out important photos. Currently, we are 
using sensor data to help us do it. In the future, we also use images processing technology to detect the 
contents of photos. Besides, current algorithm and user interface also need improvement. In addition, we plan 
to conduct a full evaluation experiment and invite more students to use this system in the near future. 
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